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This paper will include critical information for administrators and faculty in higher education for 
developing an assessment model for an online degree program. Determining the assessment process for 
the program before offering courses will ensure a smooth transition into the online environment so that 
faculty and administrators will know that the outcomes for the program are being addressed and evaluated. 
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The purpose of this paper is to provide an assessment model that has been designed and applied to a higher 
education online degree program. Too many times courses and/or programs are introduced into the online format 
without formal evaluation plans outlined to determine if the program reflects the aims and purposes of the higher 
education institution or the department. To be successful, the outcomes of the program, and the process for 
collecting them, must be established before the first course is offered. Without the assessment model in place, 
“knee-jerk” reactions will drive the program. Without the outcomes clearly delineated before the program is put in 
place, quality results cannot be obtained. 

Problem Statement 

The problem in this study was to develop an assessment model that could be utilized in an online degree in higher 
education. 

Theoretical Framework 

Accountability is the norm for all areas of work and education today, and this is especially true for higher education. 
However, designing an effective accountability system that will truly measure what needs to be measured so that 
reliable results will be provided is often times difficult to execute. Assessment projects are often times handicapped 
because the person, institution or entity does not have a clear picture of what truly needs to be assessed (Astin, 
1993). Many times organizations keep performing the same kinds of assessments even though the rationale for the 
assessment has never been established, validated, or evaluated. 

When establishing an online degree program, it is imperative that the department and the faculty identify the 
outcomes, or core values, that a student must obtain as a result of completing the required curriculum. It is not 
enough to simply give lip service to “outcome assessments”’(Astin, 1993). Bothel (2002) identified that “one of the 
most significant challenges in online course delivery is the testing and assessment component” (p. 99). He further 
said that the effectiveness of the assessment process is challenged by the unrealistic appraisal of the potential of 
online education. He stated that distance education skills bias and the limitations of online media also contribute to 
faulty evaluation methods for online courses. (Bothel, 2002). To ensure that learning outcomes are appropriate to 
the rigor and breadth of the degree or certificate awarded and that faculty and support staff are appropriately selected 
and properly utilized, administrators must identify exactly what assessment procedures will be used, how the 
program will be evaluated, and how the curriculum will be evaluated. 

Offering online courses and evaluating them is only one of the pieces to a quality online degree program. A 
vehicle must be in place to provide the support services that are critical for student success in the online 
environment. The University of Texas System created the University of Texas Telecampus (UTTC) to provide the 
support services for students, faculty, and the components that comprise the UT System. The goal of the UT 
TeleCampus in placing programs online was the creation of collaborative degrees, utilizing the best resources in 
faculty expertise from all campuses (UT TeleCampus). 

The Department of Technology at The University of Texas at Tyler (UTT) followed this model when creating 
the Master of Science Degree in Technology/HRD option UTTC offers support to the components of the UT 
System in instructional design and course development, as well as faculty training in bringing courses from the 
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lecture-based classroom to the Web-based classroom. Technological support, policy, marketing 
external communications are among the other essential support services provided to the campus 
however, the assessment of the program and/or courses is left to the discretion of the component 
degree (UT TeleCampus). 

Research Questions 

This study addressed three research questions; 

1. What comprises an assessment model to evaluate a degree program as perceived by faculty and the HRD 
Academic Advisory Council? 

2. How do students rate the quality of the course? 

3. How do students assess the services and technical components of the course? 

Overview 

The Department of Technology, in support of the mission of UTT, is a student-centered department committed to 
conducting multi-option programs pertinent to today’s changing workforce needs. With the creation of the on- 
campus Master of Science in Technology/HRD option four years ago, the explosion of the enrollment in this newly 
created program indicated the need to provide the same degree in the online environment. In the spring of 2002, the 
Department submitted a proposal to UTTC to develop the degree for the online environment over a three-year 
period. In the proposed third year of the phase-in program, additional courses that enhance the HRD option wiould 
allow students to complete the State Board of Educator Certification (SBEC) Trade and Industrial (T & 1) alternative 
teacher certification to help address the critical shortage of teachers in this discipline. 

The Trade and Industrial certification prepares Texas teachers to instruct at the secondary level in Texas public 
schools, to prepare young people to manage the dual roles of family member and wage earner, to enable them to 
gain entry-level employment in a high-skill, high-wage job and/or to continue their education. T&l certified 
teachers can teach in the areas of communications and media, construction maintenance, electronics, industrial and 
manufacturing, metal technology, personal and protective service, transportation, and career preparation systems. 
The T&l certification can be part of the MS in Technology/HRD degree or students can elect to only pursue the 
certification. The coursework necessary for the certification process also provides valuable teaching methodologies 
that can prove beneficial to any person desiring to go into the training and development field. 

The online program will be assessed throughout the three-year period through formative and summative 
measures. Astin’s model for assessment, 1-E-O, (1993) served as the basis for the assessment model that was to 
serve in the ongoing evaluation of the courses and the degree program. The Chair and the faculty of the Department 
identified the targeted outcomes for the program after the proposal for the online degree program was accepted by 
UTTC. Once the outcomes were decided, the HRD Academic Advisory Council was created to provide a 
framework for implementing the degree program and then for closing the assessment loop at the end of each year of 
course offerings by reviewing the faculty and student course assessments and support service assessments. The 
HRD Academic Advisory Council will determine recommendations for improvements based upon first year 
assessment data when it meets in the spring of 2004. Recommendations will also be entertained from the 
Departmental Advisory Committee based upon its end-of-year curriculum review. These recommendations will also 
feed into the HRD Academic Advisory Council for review and program changes/modifications. The model takes 
into consideration all aspects of the on campus program and the online program (See Eigure 1). 
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Figure 1. Department of Technology Online Degree Assessment Model 
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Summary of Program 

The Master of Science degree in Technology with the Human Resource Development (HRD) non-thesis option is a 
36 semester credit hour degree program requiring two three semester credit hour core courses. Research Techniques 
and History and Philosophy of Technology/HRD, 18 semester hours in the area of concentration of HRD and 12 
semester hours of support courses, which can be taken from the other UT components or may include the T&I 
certification courses. During the last semester of the program, the candidate must successfully complete a written 
and/or oral comprehensive examination on the coursework of the degree plan. 

Students must meet the admission requirements of the UTT graduate school as outlined in the current catalog. 
Students completing the certification courses must meet the requirements of the State Board of Educator 
Certification for T&I teachers. Current guidelines for this program can be accessed at the web site of 
http://www.sbec.state.tx.us/certreq/certreq.htm 

The online Master of Science in Technology/HRD option utilizes the resources of UTT, other component 
institutions, and UTTC. The Department of Technology developed three courses in year one of the online program, 
three courses in year two of the program and will develop three courses in the final year of the program. UTT 
utilizes existing component online UTTC courses to round out the degree requirements in the major and support 
areas. 

Assessment of Program 

The program is administered by The Department of Technology, and UTT is the degree granting institution as 
authorized by the Texas Higher Education Coordinating Board. To determine the success of the program and to 
ascertain if the students exiting the program feel that their education has been value-added, it was necessary for the 
Department to develop a formal assessment model for the program. Carneval (2001) asserted that administrators do 
not rely on assessment models to evaluate their online programs. Bothel (2002) stated that online education 
continues to grow without adequate assessment programs in place. 
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Because the research is limited in the area of online program assessment, the Department was committed to 
developing an assessment model that could be administered throughout the three year phase-in process for the 
degree program and that could be effectively used in an ongoing basis after the program was fully operational. “In 
essence, the outcomes refer to those aspects of the students’ development that the institution either does influence or 
attempts to influence through its education or practice” (Bothel, 2002, p. 38). In order to develop the assessment 
model and to determine the formative and summative evaluations, the following areas were identified: 

Input 

• Program assessment and improvement is based upon the recommendations of the HRD Academic Advisory 
Council which consists of the faculty members from the Department of Technology developing and 
teaching courses for the online environment, members from UTTC, and faculty members from the other 
disciplines on campus who have online degree programs. Currently, the Council consists of 12 individuals. 

• Each course that is developed for the online program is sent to an independent evaluator prior to 
implementation who checks the course for rigorous content, resources, and links to outside information. If 
the evaluator finds a problem with the course, the faculty member makes the recommended changes before 
the course is ever placed in the online environment. 

• At the annual Advisory meeting, recommendations based upon faculty assessment, student assessment, 
student service assessment, and Departmental Advisory Committee input will be presented. Decisions for 
program and course improvement will be determined at that time. 

Environment 

Course Assessment: 

• When the course is first taught, the faculty member developing and teaching the course, conducts a 
continuous evaluation of course content, comprehension, and logistics problems that occur. 

• Standardized course assessment results will be reviewed. This provides consistency in evaluation of all the 
courses being offered for the degree. 

• Assessment of student evaluation regarding the quality of the course will be conducted. 

• An assessment of support services provided through the UTTC will be conducted to determine if students 
are being provided the services that they need to be successful in the online environment. 



Outcomes 

• Statistical analysis will be conducted to determine the outcomes of the faculty course assessment, the 
student course assessment, and the student support services assessment. 

• Recommendations from the Departmental Advisory Committee, based upon the Departmental Assessment 
Model, will be reviewed to keep the on-campus program consistent with the online program. These 
recommendations will be submitted to the HRD Academic Advisory Council. 

Conclusions and Contributions to HRD: 

The purpose of this paper was to provide an assessment model that can be used to provide formative and summative 
evaluations of an online degree program. Too many times online courses are introduced without formal evaluation 
studies conducted to determine if the courses are successful or if students leave the institution with value-added to 
their education. However, designing an effective accountability system that will truly measure what needs to be 
measured so that reliable results will be provided is often times difficult to perform. Assessment projects are often 
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times handicapped because the person, institution or entity does not have a clear picture of what truly needs to be 
assessed (Astin, 1993). 

Based upon the faculty evaluation process, the three faculty members who developed courses will be making 
their recommendations for changes/improvements to the HRD Academic Advisory Council in the spring of 2004. 
Those changes, when approved by the Council, will become part of the revision process before the courses are 
offered for the second time in the spring of 2005. Based upon student course evaluations, 84% of the students 
strongly agreed that they would recommend this online course to their peers. 90% agreed that the courses were well 
organized and clearly presented. These results will also be tabulated into table format and presented to the Advisory 
Council for their recommendations. 89% of the students provided positive comments on the student support 
services questionnaire. 84% indicated that the course navigation was simple and user-friendly. 71 % reported that 
they believed the online course to be comparable or better than the on campus courses and 84 % stated that their 
overall experience with online education was positive. 

With the formative assessment process in place for the first three courses, the faculty and Chair of the 
Department of Technology will have quantitative results to share with the HRD Academic Advisory Council when it 
meets in the spring semester of 2004. Based upon the faculty evaluations, the student course evaluations, and the 
student support services evaluation, and considering the recommendations for curriculum improvement from the 
Technology Advisory Committee, the assessment model is in place and functioning to provide constructive 
improvement and refinement for the MS in Technology/HRD option. 

The Department of Technology has developed a model for assessing an online degree program based upon the 
model published and validated by Astin (1993). The assessments for the first three courses are currently being 
analyzed and the results will be reported to the HRD Academic Advisory Council in the spring of 2005. 
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